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METHOD A>m APPaPfiTTT.q TTOR M EASURING SIMILARITY 
USING MATCHING PIXEL COUNT 

. 5 Technical Field: 

The present invention relates to digital image processing 
and, more particularly, to a method and apparatus for measuring 
similarity between pixels in two different images. 

10 

Backcrround Art : 

Stereo disparity estimation is one of the most active 
research areas in the field of three-dimensional or stereo 
15 vision. Stereo disparity is the spatial offset or displacement 

between two matching pixels, where one pixel is in a reference 
image, the other pixel is in a search image, and both pixels 
correspond to the same point in physical space. The reference 
and search images are the images viewed by the left and right 

2 0 eyes, respectively. 

In order to determine the pixel in the search image which 
matches a given pixel in the reference image, the coordinates 
of the pixels in the two images must first be established. A 
:[;^5 similarity measure is then computed between pixels in a window 

of predetermined size centered at the given pixel in the 
reference image (i.e., the reference pixel) and pixels that are 
candidates to match the reference pixel in a window of the same 
size centered at each candidate pixel in the search image. The 

3 0 pixel in the search image that yields the greatest value of the 

similarity measure is considered to be the pixel that matches 
the reference pixel . 

A disparity map for the reference image is obtained by 
35 computing the similarity measure between a reference pixel and 

each candidate matching pixel in a search range for every pixel 
in the reference image. Generation of the disparity map is 
greatly simplified by imposition of the "epipolar constraint" 
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on the candidate matching pixels, whereby the candidate 
matching pixels are limited to those which lie on the same 
horizontal line as the reference pixel. 

Figure 1 illustrates matching subject to the epipolar 
constraint. To determine the pixel in the search image that 
most closely matches the starred reference pixel, the 
similarity measure is computed for a window centered at the 
reference pixel and a window in the search image centered at 
a candidate matching pixel as the window in the search image 
is moved along a horizontal line within a search range. The 
candidate search pixel that yields the greatest value of the 
similarity measure is considered to be the pixel that matches 
the reference pixel . The spatial displacement between the 
matching pixel from the reference pixel is the stereo disparity 
for the reference pixel . 



Similarity measures typically used to determine matching 
pixels include the sum of squared differences (SSD) , the sum 
of absolute differences (SAD) , and the normalized cross 
correlation (NCC) . For each of these similarity measures, the 
contribution of a given search pixel to the disparity map 
depends on the light intensity at the given search pixel. The 
disparity boundary between a region within which there are 
large variations in light intensity therefore tends to extend 
into any adjacent region within which there are small 
variations in light intensity. This phenomenon, known as 
"boundary overreach", generates misleading disparity values 
near the boundary of an object. 

Disclosure of Invention: 



An object of the present invention is accordingly to 
suppress boundary overreach in the estimation of stereo 
disparity by providing a similarity measure based on a matching 
pixel count ("MPC") , the number of search pixels having similar 
intensity values, rather than on the light intensity at each 
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of the search pixels. Another object of the present invention 
is to provide an apparatus for estimating stereo disparity that 
efficiently implements the above method in real time. 

One embodiment of the present invention is a method for 
determining the stereo disparity between a reference image and 
a search image for a reference pixel in the reference image, 
the method comprising the steps of: 

(a) calculating a similarity measure between a reference 
window in the reference image, which reference window includes 
a set of pixels centered at the reference pixel, and each of 
a plurality of search windows in the search image, each of 
which search windows has the same shape as the reference window 
and is displaced from the reference window within a 
predetermined search range, wherein a matching pixel count, the 
number of pixels in the reference window at which the light 
intensity is similar to the light intensity at the 
corresponding pixels in the search window, is used as the 
similarity measure between the reference window and the search 
window; and 

(b) determining a displacement between the reference 
window and a search window which yields the greatest value of 
the similarity measure as the stereo disparity for the 
reference pixel. 

Another embodiment of the present invention is an 
apparatus for determining the stereo disparity between a 
reference image and a search image for a reference pixel in the 
reference image, the apparatus comprising: 

(a) first means for calculating a similarity measure 
between a reference window in the reference image, which 
reference window includes a set of pixels centered at the 
reference pixel, and each of a plurality of search windows in 
the search image, each of which search windows has the same 
shape as the reference window and is displaced from the 
reference window within a predetermined search range, wherein 
a matching pixel count, which is the number of pixels in the 
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reference window which are similar in intensity to 
corresponding pixels in a search window, is used as the 
similarity measure between the reference window and the search 
window; and 

(b) second means for determining a displacement between 
the reference window and a search window which yields a largest 
value of the similarity measure as the stereo disparity for the 
reference pixel, 

wherein R(x,y) denotes the reference pixel, the reference 
window includes Wx*Wy pixels centered at R{x,y), Wx and Wy 
being predetermined numbers, each of the search windows 
includes Wx*Wy pixels centered at L(x+d, y) which is a pixel 
in the search image , d ranging from 0 to a predetermined number 
Sr, and 

the first means includes : 

(al) a P-unit for calculating P(x,y,d), where: 

P(x,y,d) = 1, if abs (BR(x,y) -Bj„(x+d,y) ) s Th and 

P(x,y,d) = 0, otherwise, 
where B^{yi,Y) and BL(x+d,y) denote the intensity values at 
R{x,y) and L{x+d,y), respectively, and Th is a predetermined 
threshold value; 

(a2) a P-buffer for storing P(x,y,d) values 
calculated by the P-unit; and 

(a3) third means for determining MPC(x,y,d) values 
for d=0 to Sr as follows: 

MPC(x,y, d) =Y^P{x,y, d) 



where w denotes the reference window and a search window; and 
the second means includes means for selecting a d value 
which yields a largest iyiPC(x,y,d) value as the stereo disparity 
for R{x,y) . 



Brief Description of Drawings 
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Figure 1 illustrates matching subject to an epipolar 
constraint . 

Figure 2 illustrates the comparative performance of the 
SAD and MFC similarity measures. 
5 Figures 3A and 3B illustrate redundant operations involved 

in determining the MPC similarity measure. 

Figures 4A and 4B illustrate a method of determining the 
MPC similarity measure from which redundant operations have 
been eliminated. 

10 Figure 5 is a flow chart for determining the MPC 

similarity measure in accordance with an embodiment of the 
present invention. 

Figure 6 shows an apparatus for determining a disparity 
map D(x,y) in accordance with an embodiment of the present 
15 invention. 

Figure 7A illustrates the internal structure of the P- 
buffer of the apparatus shown in Figure 6. 

Figure 7B illustrates the internal structure of the V- 
buffer of the apparatus shown in Figure 6. 

2 0 Figure 8A illustrates the internal structure of the P-unit 

of the apparatus shown in Figure 6 

Figure 8B is a circuit diagram of the D_P unit of the P- 
unit illustrated in Figure 8A. 

Figure 9 illustrates the internal structure of one of the 
25 MPC-units of the apparatus shown in Figure 6. 

Figure 10 is a logic diagram of the MPC-unit whose 
internal structure is illustrated in Figure 9. 

Figure 11 illustrates the internal structure of the 
Max_MPC selector of the apparatus shown in Fig\ure 6 . 

30 

Best Mode for Carrying out the Invention: 

In the method described and claimed in the present 
application, a matching pixel count {"MPC"} is used as a 

3 5 measure of the similarity between a reference window in a 

reference image and each of a plurality of search windows in 
a search image. The contribution of a given matching pixel in 
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the search image to the similarity measure is taken to be the 
same irrespective of the intensity at the given matching pixel 
in order to ensure an accurate result near the boundary of the 
image. MPC is computed by counting the number of pixels in a 
5 window in one image whose corresponding pixels in another image 

have similar intensity values. 



When the reference and search windows satisfy the epipolar 
constraint, MPC{x, y, d) denotes the similarity between a 
10 reference window centered at R(x, y) and a search window 

centered at L(x+d, y) , where R(x, y) is a pixel in the 
reference image located at (x, y) and L(x+d, y) is a pixel in 
the search image located at (x+d, y) . MPC(x, y, d) is defined 
by: 

15 

MPC{x,y,d) =Y^P{x,y,d) (1) 



where P{x,y,d) =1, if abs [B^ (x, y) -B^ (x+d, y) ] s Th 
= 0, otherwise; 

abs ( • ) denotes the absolute value; B(j{x,y) and El (x+d, y) denote 
20 the intensity at R(x, y) and L(x+d, y) , respectively; W denotes 

a matching window of size (Wx * Wy) , a group of pixels centered 
at either R(x, y) in the reference image or L(x+d, y) in the 
search image; and Th is a predetermined threshold value. 



25 P{x, y, d) is thus set equal to one if intensity values 

at corresponding pixels, i.e., R(x, y) and L (x+d, y) , in the 
two images are similar, and zero otherwise. To determine the 
disparity for a reference pixel R(x, y) , iy[PC(x, y, d) is 
computed for all values of d in a search range {d=0 to Sr) and 

30 the disparity is defined to be the value of d that yields the 

greatest value of MPC (x, y, d) . 



Figure 2 illustrates a comparison of SAD and MPC 
similarity measures. A comparison of the intensity values of 
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pixels in the reference image with the intensity values of 
pixels in the search image shows that the disparity of the 
three upper pixels in the reference image, which have intensity 
values of 10, 200, and 10, is zero and the disparity of the six 
lower pixels in the reference image, which have intensity 
values of 150, 80, 60, 70, 200 and 250, is one. 



To determine the disparity of the pixel with an intensity 
value of 80 (the "reference pixel"), a similarity measure 

10 (either SAD or MPC) between a window in the reference image 

(the "reference window") and each window in the search image 
("search window") is computed and the search window most 
similar to the reference window is selected. The displacement 
between the reference window and the selected search window is 

15 the disparity between the two images for the reference pixel. 

For sake of simplicity, only two search windows are shown in 
Figure 2 . 

In the SAD method, SAD (3, 2,0) and SAD (3, 2,1) are given by: 
20 SAD(3,2,0)= abs (10-10) +abs (200-200) +abs (10-10) 

+abs (140-150) +abs (150-80) + abs (80-60) 
+abs (80-70) +abs (7 0-2 00) +abs (2 0 0-250) 
=2 90 and 

SAD (3, 2, 1) = abs (200-10) +abs (10-2 0 0) +abs (2 50-10) 
25 +abs (150-150) +abs (80-80) +abs (60-60) 

-fabs (70-70) +abs (200-200) +abs (250-250) 
= 620. 

In SAD (3, 2,0) and SAD (3, 2,1), 3 and 2 are the x and y 
coordinates, respectively, of the reference pixel and 0 or 1 
3 0 is the disparity between reference and search windows for which 

the SAD is computed. The disparity for the reference pixel in 
the SAD method is the displacement that yields the smallest SAD 
value {i.e., 0) . 



In the MPC method, MPC (3, 2,0) and MPC (3 , 2 , 1 ) are given by: 
MPC(3,2,0) = 1+1+1+0+0+0+0+0+0=3 
MPC(3,2,1) = 0+0+0+1+1+1+1+1+1=6 
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The disparity for the reference pixel in the MPC method is the 
displacement that yields the largest MPC value (i.e., 1). 

In this example, the SAD measure yields a misleading 
5 disparity value, because the upper three pixels in the 

reference window, which exhibit a large variance in their 
intensity values, have a greater effect on the similarity 
measure than do the lower six pixels, which exhibit a small 
variance . 

10 

Figures 3A and 3B illustrate redundant operations which 
are inherent in the computation of the MPC measure. Despite 
the epipolar constraint, the amount of computation required to 
determine MPC values for all pixels in the reference image is 
15 proportional to (Ix * ly) * (Wx * Wy) * Sr, where (Ix * ly) is 

the size of an image, {Wx * Wy) is the size of a window, Wx and 
Wy being odd numbers, and Sr is a search range. The enormous 
amount of computation required is due to the redundant 
operations which are inherent in the MPC method. 

20 

As shown in Figure 3A, after the MPC value between R(x, 
y) and L{x+d, y) has been determined, the computation 
corresponding to the overlapping area of the two horizontally 
displaced windows would be redundant when determining the MPC 
25 value between R(x+1, y) and L(x+l+d, y) . Similarly, when 

determining the MPC value between R(x, y+1) and L(x+d, y+1) , 
the computation corresponding to the overlapping area of two 
vertically displaced windows in Figure 3B would be redundant. 

30 By using buffers to store the results of computation, 

these redundant operations can be avoided so that the amount 
of computation can be reduced to (Ix * ly) * Sr, irrespective 
of the size of a window. 

3 5 Figures 4A and 4B illustrate a method of computing MPC 

values from which these redundant computations have been 
eliminated. For sake of simplicity, only pixels in the 
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reference image are shown. V(x, y, d) represents a matching 
pixel count between two vertical segments which are a group of 
Wy pixels centered at R(x, y) and L(x+d, y) . MPC(wx, y, d) is 
computed by summing V values as follows: 

MPCixw.y, d) = V{wx+i,y,d) (2) 



For X larger than wx, MPC(x,y,d) can be computed from a 
previously computed MPC value (i.e., MFC (x-1 , y, d) ) as follows: 

MPC{x,y, d) =MPCix-l,y, d) +V{x+wx,y, d) -V{x-l.-wx , y , d) (3) 

10 

In Equations (2) and (3) , wx and wy denote distances from the 
center to the boundary of a window in the horizontal and 
vertical directions, respectively: 

wx= {Wx~l) / 2 and wy= {Wy-1) /2 (4) 

15 

In a similar manner, a first V value in a column, 
V(x,wy,d) , is computed by summing P(-) for a vertical segment: 

20 V{x,wy,d) p{x,wy+i,d) (5) 



where P(x, y, d) has the value one if values of R(x, y) and 
L(x+d,y) are similar and has the value zero otherwise as 
defined in Equation. (1) . For values of y greater than wy, 
V(x,y,d) can be computed by using a previously computed V 
25 value, V(x,y-l,d) : 
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V(x,y,d) =V{x, y-1, d) +P{x, y+wy, d) -P {x, y-l-wy, d) (6) 

At the beginning of stereo disparity estimation for a 
reference image and a search image, V(x,wy,d) , which represents 
the similarity between vertical segments centered at R (x, wy) 
5 and L(x+d, wy) , is computed for all values of x and d (i.e., 

from x=0 to x=Ix-l and from d=0 to d=Sr) according to Equation. 
(5) . In other words, as many as (Sr+1) V values, V(x,wy,0), 
V(x,wy,l), ... , V(x,wy,Sr), are computed for each x. The 
computed V values are stored in a V buffer which can store 
10 Ix*(Sr+l) V values. 

In Equation (2), MPC(wx,y,d) represents the similarity 
between a window centered at R{wx,y) in the reference image and 
a window centered at L(wx+d,y) in the search image. A buffer 
15 for storing a MPC value is initialized with a sum of 

V(wx+i,y,d) values for I = -wx to wx. The initialization of the 
MPC buffer is done for a (wx) th pixel in each row. 

For values of x greater than wx, MPC(x,y,d) is computed 
20 by using MPC (x- 1 , y , d) , the MPC for the previous pixel in the 

same row, which is stored in the MPC buffer, as shown in 
Equation (3) and Figure 4A. Specifically, V(x+wx,y,d), a V 
value for a vertical segment newly added to the window, is 
added to the previously computed MPC value and V (x-wx-1 , y, d) , 
25 a V value for a vertical segment excluded from the window, is 

subtracted from the previously computed MPC value. 

Similarly, for values of y greater than wy, V(x,y,d) can 
be computed without redundancies in the vertical direction. 
3 0 P(x,y+wy,d) indicating matching between pixels added to the 

vertical segment is added to the previously computed V value, 
V(x,y-l,d), stored in the V buffer, and P (x, y- 1 -wy , d) 
indicating matching between pixels excluded from the vertical 
segment is subtracted from V(x,y-l,d) . 



35 
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In summary, after initializing the V buffers with the V 
values for vertical segments centered at each pixel in the 
(wy) th row and initializing the MPC buffer with the MPC value 
for a (wx) th pixel in a row, other MPC values may be determined 
without redundant operations, as indicated by Equations (3) and 
(6) . In doing this, search range for each pixel should be 
identical. For a pixel in the reference image R(x,y), if a 
best matching pixel among all the pixels of the search image 
in the search range is L(x+d„a^, y) , d^^^ is the stereo 
disparity for R(x,y). 

Figure 5 illustrates the MPC matching algorithm without 
redundancies for implementing the above equations in an 
efficient manner. Inputs to this algorithm are: I) the 

intensity values of pixels R(x,y) and L(x,y} in the reference 
and search images, respectively, where x ranges from 0 to Ix-1 
and y ranges from 0 to Iy-1; ii) Wx and Wy representing the 
size of a matching window, Wx, Wy being odd integers; and iii) 
the search range Sr. An output from this algorithm is a 
disparity map containing a disparity for each pixel, DM(x,y), 
where x ranges from wx to Ix-wx and y ranges from wy to ly-wy. 



In Step S20, y is initialized to wy. Then, in Step S30, 
25 initial V values, V(x, wy, d) for x=0 to Ix-1, d=0 to Sr, are 

computed and the results are stored in the V buffers. Since 
the X coordinate of a pixel cannot exceed Ix-1 and R(x+d, y) 
is used in the computation, x ranges from 0 to Ix-l-d, instead 
of Ix-1. 

30 

In Step SlOO, the MPC values are either initialized or 
updated, depending on the value of x, and the V values are 
updated. This step is executed for each value of x, y and d 
(x=wx to Ix+wx, y=wy to Iy-wy-1, d=0 to Sr) . If x=wx, Equation 
35 (3) is executed to generate the first MPC value in the row. 

If the value of x is less than Ix-d-wx and greater than wx, 
Equations (3) and (6) are executed. If x is greater than or 
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equal to Ix-d-wx, only the V values are updated. After y is 
incremented, the V values updated in SlOO are used to compute 
the MPC values for the next row. 



Steps S40, S50, S60, S70, S80, S90, and S95 repeat Step 
SlOO for the given ranges of x, y and d. 

Steps S130 and S140 determine the value of d that yields 
the greatest MPC value as the disparity for the reference 
pixel. After all the steps have been completed, a depth map 
DM(x,y) (for x= wx to Ix-wx, y=wy to ly-wy) is obtained. 

Figure 6 illustrates an apparatus for determining the 
disparity values in accordance with the present invention. The 
apparatus includes a P-buffer 100, an MPC processor 200 and a 
V-buffer 300 in which Wx and Wy are set to seven, so that the 
size of a window is 7x7, and Sr is set to 63. Inputs to the 
apparatus are pixel values of the reference and search images 
and Th for determining whether two pixels are similar. 
Synchronization signals, HSync and VSync are also input to the 
apparatus. Outputs from this apparatus are the maximum 
MPC(x,y,d) value (MaxV) for pixel R(x,y) in the reference image 
and disparity DM(x,y) which yields the maximum MPC value. 

The MPC processor 200 includes a P-unit 210, MPC-units 
220, a Max_MPC selector 230 and a control unit 240. The MPC 
processor 200 includes (Sr+1) MPC units 220, so that for each 
reference pixel R{x, y) , (Sr+1) MPC values for a search range 
are computed in parallel. Pixel values R(x,y) and L(x+d,y) are 
fed to the P-unit 210, where P(x,y,d) is determined according 
to Equation (1). In the P-unit 210 , (Sr+1) P (x, y, d) -values for 
a given (x,y) (d=0 to Sr) are determined in parallel. The P- 
values are provided to the P-buffer 10 0 and stored therein. 

The P values stored in the P-buffer 10 0 are coupled to the 
(Sr+1) MPC-units 220, where MPC(x,y,d) and V(x,y,d) are 
computed. A vector Pv(d), a set of P values for a same d 
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value, are input to the MPC-units 220 from the P-buffer 100 

V(x,y,d) computed at the MFC units 220 (Vo in the figure) is 
input to the V-buffer 300 and stored therein. Each of the MPC- 
units 220 computes the MPC value for each d by using a V value 
5 computed therein and V values retrieved from the V-buffer 300. 

To do this, Va and Vi , V values for vertical segments added to 
and excluded from a matching window, respectively, are provided 
from the V-buffer 300 to each of the MPC-units 220. 



10 Max_MPC selector 23 0 receives (Sr+1) MPC values for a 

pixel R(x,y) from the MPC-units 220 and determines a largest 
one (MaxV) to provide DM(x, y) . 

Figure 7A illustrates the internal structure of the P- 

15 buffer 100, which includes (Sr+1) parallel (Ix*Wy) bit shift 

registers. The P-buffer stores the P{x,y,d) values (identified 
as Pi in the figure) provided from the P-unit 210 in the MPC 
processor 200 and timely provides them to the (Sr+1) MPC-units 
22 0. Each of the blocks identified as P_R(x,y) stores 64 

20 (i.e., (Sr+1) ) 1-bit P (x, y, d) -values and the P (x, y, d) -values 

are shifted in synchro- nization with a system clock, as 
indicated by the arrows. In Figure 7A, each row of blocks 
corresponds to a row of an image and each column of blocks 
corresponds to a vertical segment in a window (i.e., Wy 

25 consecutive pixels in a column) . Thus, P(x,0,d) to P (x, Wy-1 , d) 

p-values for pixels included in the same vertical segment are 
provided from the left part of the P-buffer 100, where each of 
P(x,0,d) to P(x,Wy-l,d) represents 64 P-values (either 0 or 1) 
for d=0 to 63 . These values are realigned according to the d 

30 values into Sr+1 Pv(d) so that Wy (in this example, 7) P values 

for a same d constitute a vector Pv(d) . Since the P-buffer 100 
is constructed with shift registers, desired results are 
provided when the registers are fully occupied by P values, 
that is, Ix * Wy clocks after a data is first fed to the P- 

35 buffer. Then, Pv(d)s are provided from the P-buffer 100 

consecutively. Each Pv(d) from the P-buffer 100 is fed back 
to each MPC-unit in the MPC-processor 200 to be used in 
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Figure 7B shows the internal structure of the V-buffer 
300. The V-buffer 300 temporarily stores V(x,y,d) values which 
5 are computed at the (Sr+1) MPC-units 220. Each block 

identified as V_R ( I) stores 64 (d=0 to 63) 3-bit V(x,y,d) 
values. The data in each block is shifted to a next block in 
synchronization with a clock signal. Data Stored in V R(Wx) 
and V_R(0) are provided as outputs from the V-buffer 3 00, which 

10 correspond to V values for vertical segments added to and 

subtracted from a matching window, respectively. In other 
words, each of Vw(0) to Vw(63) represents a V value for a 
vertical segment added to a window for d=0 to 63 in the 
computation of MPC . Similarly, each of Vo(0) to Vo(63) 

15 represents a V value for a vertical segment excluded from a 

window for d=0 to 63. The V-buffer 300 also has initial delay 
of (Ix-Wx) clock cycles. Outputs from the V-buffer 300 are 
provided to the MPC-unit 220 to be used in computing MPC values 
according to Equation (3) . 

20 

Figure 8A shows an internal structure of the P-unit 210 
included in the MPC-processor 200. Inputs to the P-unit 210 
are R, S and Th where R and S denote pixel values of reference 
and search images, and Th denotes the threshold for determining 

25 P(x,y,d) . Assuming that a pixel value ranges from 0 to 255 (2'- 

1) , 8 bits are assigned to each R and S input. The P-unit 210 
receives (Sr+1) pixel values in a search image for each pixel 
in a reference image and determines P(x,y,d) for them. A 
processing unit denoted as D_P (d) , d=0 to Sr, determines 

30 P(x,y,d) for the reference pixel ( i.e., the pixel in the 

reference image) and a pixel in the search image displaced from 
the reference pixel by d. 

Figure SB is a circuit diagram of the D_P unit 211. Full 
35 adders (FA) 211-1 in the left part subtract S from R. Carry 

generators (CG) 211-3 in the right part provide a carry, 
denoted as D, depending upon the sign of (Th-abs (R-S) ) . If Th 
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is greater than the absolute value of (R-S) , D is 1 . If (R-S) , 
the input to the XOR gates 211-2, is a positive number, the 2's 
complement of the inputs (i.e., -(R-S)) is provided by the 
gates 211-2. If (R-S) is a negative number, (R-S) is provided 
by the XOR gates, so that the absolute value of (R-S) can be 
subtracted from Th . Since there are (Sr+1) D_P (d) units 211 
in the P_units 210, (Sr+1) P(x,y,d) values may be calculated 
in parallel. In the P-unit 210 shown in Figure 8A, a pixel 
value in a reference image is used only once, while a pixel 
value in the search image is used (Sr+1) times as it is shifted 
through D_R(d) . D_R(d) denotes the registers that store pixel 
values of the search image. A pixel value in each D_R(d) is 
shifted left on each clock. As the P-unit 210 includes the 
shift registers, it provides an output continuously after an 
initial delay of (Sr+1) clock cycles. 

Figure 9 illustrates the internal structure of the MPC- 
unit 220 in the MFC processor 200. The MFC-unit 220 is divided 
by its function into a V_MP counter 221 and a V_MP update unit 
222 for calculating V(x,y,d) according to Eqs . (5) and (6), 
respectively, and a W_MP count and update unit 223 for 
calculating MPC(x,y,d) according to Eqs. (2) and (3) . The V_MP 
counter 211 is used to provide V(x,y,d) by summing P values 
inputted in an initial stage of the disparity map computation. 

After the initial stage, V (denoted as Vi in the figure) is 
provided to the V_MP update unit 222 from the V-buffer 300. 
Then, V_MP update unit 222 calculates V(x,y,d) without 
redundant operat i on . 

Figure 10 depicts a logic diagram of the MPC-unit 220. 
The V_MP counter 221 receives Wy P values in an initial stage, 
adds them at four full adders (FAs) included in the V_MP 
counter 221, thereby providing a 3 -bit binary number V(x,y,d) . 
This 3-bit value (Vo(2), Vo(l), Vo(0)) is provided to the V- 
buffer 300 through a MUX A 225. C2 control input makes the MUX 
A 225 select an output from the V_MP counter 221 until an 
output from the V_MP update unit 222 becomes valid, V(x,y,d) 
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values input to the V-buffer 300 are provided to the Va input 
port of the MPC-unit 220 after (Ix-Wx) clock cycles and to the 
Vi input port after Ix clock cycles. V_MP update unit 222 
begins to calculate V(x,y,d) without redundancy according to 
5 Equation (6) by adding one to Vi or subtracting one from Vi at 

the full adders 222-3, depending on the values of Pv ( 0 ) and 
Pv(6) . Specifically, if Pv{0) is equal to Pv(6), 0 (000 in 
binary) is added to Vi ; Pv(0) is 1 and Pv(6) is 0, -1 (111 in 
binary) is added to Vi ; and Pv(0) is 0 and Pv(6) is 1, 1 (001 
10 in binary) is added to Vi . Then, the C2 control input enables 

the MUX A 22 5 to select the outputs from the V_MP update unit 
222 . 

Vi and Va, provided from the V-buffer 300 to the MPC-unit 
15 22 0, are used to calculate MPC values according to Equations 

(2) and (3) . Full adders 223-1 on the left part of the W_MP 
count and update unit 223 calculate Va-Vi . The result is added 
to a previous MPC value in the adders 223-2 on the right part 
of the W_MP count and update unit 223 in order to provide an 
20 updated MPC value. MUX B 224 selects O's and provides them to 

the W_MP count and update unit 223 until Vi from the V-buffer 
3 00 becomes valid. As a result, until Vi becomes valid, only 
Va values are summed in the unit 223 to provide MPC(wx, y, d) 
according to Equation (2) . After Vi from the V-buffer becomes 
25 valid, MPC is calculated according to Equation (3) . 

Figure 11 shows the internal structure of the Max_MPC 
selector 230 included in the MPC-processor 200 in Figure 6. The 
Max_MPC selector 23 0 compares MPC values provided from the 
30 (Sr+1) MPC-units 220, selects a maximum value MaxV and provides 

a displacement DM(x,y) corresponding to the maximum value. 
Basic units for constructing the Max__MPC selector 23 0 are 
C&A(n) cells 231 each of which compares its inputs and provides 
the larger one and its location information. 



Figure 12 is a logic diagram of the C&A(n) cell. In the 
C&A cell 231, carry generators 231-1 provides 0 if lb is 
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smaller than la and 1 otherwise so that MUXes 231-3 select and 
provide the larger value. MUXes 231-2 produce the location 
information (S (n) , ... , s(0)) corresponding to the larger 
value in response to the location information from a previous 
5 stage, i.e., CSA(n-l), and the carry value. The number of 

MUXes in the C&A cell varies depending on n, that is, the 
number of MUXes 231-2 is n. The MUXes 231-2 select one between 
Sa and Sb, the location information of a previous stage. In 
Fig. 11, going further to the right, number of bits inputted 
10 as Sa and Sb to C&A cell increases. The output from the CEJ^ 

cell of the last stage is a maximum MPC value for a given pixel 
R(x,y) and its disparity. 

While the present invention has been described in terms 
15 of particular embodiments, it will be apparent to those of 

ordinary skill in the art that various changes and 
modifications may be made without departing from the spirit and 
scope of the invention defined in the following claims. 



20 
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1. A method of determining a stereo disparity between a 
reference image and a search image for a reference pixel in the 
5 reference image, said method comprising the steps of: 

(a) calculating a similarity measure between a reference 
window including a set of pixels centering on the reference 
pixel and each of a group of search windows in the search image 
which is of a same shape with the reference window and 

10 displaced from the reference window within a predetermined 

search range, wherein a matching pixel count, which is the 
number of pixels in the reference window which are similar in 
intensity to corresponding pixels in a search window, is used 
as the similarity measure between the reference window and said 

15 search window; and 

(b) determining a displacement between the reference 
window and a search window which yields a largest similarity 
measure as the stereo disparity for the reference pixel . 

2 0 2. A method of determining a stereo disparity as defined in 

Claim 1, wherein R(x,y) represents the reference pixel, the 
reference window includes Wx*Wy pixels centering on R(x,y) , Wx 
and Wy being predetermined numbers, each of the search windows 
includes ■Wx*Wy pixels centering on L(x+d, y) which is a pixel 
2 5 in the search image, d ranging from 0 to a predetermined number 

Sr, and 

said step (a) includes: 

(al) calculating P{x,y,d) values as follows: 

30 P{x,y,d) = 1, if abs (BR(x,y) -BL(x+d,y) ) s Th 

= 0, otherwise, 

where BR{x,y) and BL(x+d,y) represent intensity values of R(x, 
y) and L(x+d, y) and Th is a predetermined threshold; and 
35 (a2) determining MPC(x,y,d) values for d=0 to Sr as 

follows : 
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MPC{x,y, d) P{x,y, d) 



wherein w represents the reference window and the search window 
centering on L(x+d,y); and 

said step (b) includes selecting a d value which yields 
5 a largest MPC(x,y,d) value as the stereo disparity for R(x,y) . 

3. An apparatus for determining a stereo disparity between 
a reference image and a search image for a reference pixel in 
the reference image, said apparatus comprising: 

10 (a) first means for calculating a similarity measure 

between a reference window including a set of pixels centering 
on the reference pixel and each of a group of search windows 
in the search image which is of a same shape with the reference 
window and displaced from the reference window within a 

15 predetermined search range, wherein a matching pixel count, 

which is the number of pixels in the reference window which are 
similar in intensity to corresponding pixels in a search 
window, is used as the similarity measure between the reference 
window and said search window; and 

2 0 (b) second means for determining a displacement between 

the reference window and a search window which yields a largest 
similarity measure as the stereo disparity for the reference 
pixel , 

wherein R(x,y) represents the reference pixel, the 
25 reference window includes Wx*Wy pixels centering on R(x,y), Wx 

and Wy being predetermined numbers, each of the search windows 
includes Wx*Wy pixels centering on L(x+d, y) which is a pixel 
in the search image, d ranging from 0 to a predetermined number 
Sr, and 

30 said first means includes: 

(al) a P-unit for calculating P(x,y,d) values as 

follows : 



P(x,y,d)= 1, if abs (Br (x,y) -BL(x+d,y) ) s Th 
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where BR(x,y) and BL(x+d,y) represent intensity values of 
R(x,y) , and L(x+d,y) and Th is a predetermined threshold value; 

(a2) a P-buffer for storing P(x,y,d) values from said 

P-unit ; 

{a3) third means for determining MPC(x,y,d) values 
for d=0 to Sr as follows: 

MPC{K,y, d) =52^^^'^'^) 



where w represents the reference window and the search window 
centering on L(x+d,y); and 

said second means includes means for selecting a d value 
which yields a largest MPC(x,y,d) value as the stereo disparity 
15 for R (x, y) . 



4. An apparatus as defined in Claim 3, wherein said third 
means includes (Sr+1) MPC-units, each of which determines 

20 MPC(x,y,d) for each d value. 

5. An apparatus as defined in Claim 4, wherein each of said 
MFC-units includes: 

means for determining V(x,y,d) values which is represented 
25 as follows: 

V{x,y,d)=Yl Pix,y+i,d) 



30 



where wy is (Wy-l)/2; 

means for generating a MPC(x,y,d) value by using V(x,y,d) 
values as follows: 
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MPCix, y, d) ^ V{x+i,y,d), if x=wx and 



MPC(x,y,d)= MPC(x-l,y,d)+V(x+wx,y,d) -V(x-l-wx,y,d) , if 

x>wx, 

where wx is (Wx-l)/2. 

6. An apparatus as defined in Claim 5, further comprising a 
V-buffer for storing the V{x,y,d) values from said V 
determining means and providing the stored V(x,y,d) values to 
said MPC generating means. 

7. An apparatus as defined in Claim 6, 
wherein said V determining means includes: 

a V_MP counter for determining V(x,y,d) values for 
by summing P values as follows: 



V{x,y,d)=^ P{x,y+i,d) 



a V_MP update unit for determining V(x,y,d) values 
by using V(x,y-1, d) and P values as follows: 

V{x,y,d) =V(x,y-l,d) +P(x,y+wy, d) -P (x, y-l-wy, d) ; and 

a multiplexor for selectively providing the V(x,y,d) 
value from the V_MP counter if y=wy and the V(x,y,d) value from 
the V_MP update unit if yswy; and 

said MPC generating means includes: 

a W_MP count and update unit for generating a 
MPC(x,y,d) value by .using V(x,y,d) values; and 

a multiplexor for selectively providing V(x,y,d) 
values from the V-buffer or 0 to the W_MP count and update unit 
as the V(x-l-wx,y,d) value. 



8. An apparatus as defined in Claim 7, wherein 
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said V_MP counter includes a plurality of full adders; 
said V_MP update unit includes: 

logic gates for providing P (x,y+wy, d) -P (x, y-l-wy, d) ; 

and 

5 full adders for adding the output from the logic 

gates to V(x,y-l,d), thereby providing V(x,y,d); and 
said W_MP count and update unit includes: 

means for deciding V (x+wx, y, d) -V (x- 1-wx, y, d) ; and 
means for adding the output from said deciding means 
10 to MFC (x-l,y,d) . 

9. An apparatus as defined in Claim 3, wherein said P-unit 
includes: 

(Sr+1) D_R units each of which stores L(x+d,y) values for 
15 each d; and 

(Sr+1) D_P units which provides (Sr+1) P(x,y,d) values for 
d=0 to Sr simultaneously in response to R(x,y) and (Sr+1) 
L(x+d, y) values from the D_R units. 

20 10. An apparatus as defined in Claim 9, wherein the D_P unit 

includes : 

means for calculating (Bl (x+d, y) -B^ (x, y) ) which includes 
a plurality of full adders; 

means for calculating an absolute value of {BL(x+d,y)- 
25 BR(x,y)) which includes a plurality of exclusive OR gates; and 

means for subtracting the absolute value from Th and 
providing 0 or 1 depending on the result of the subtraction, 
which includes a plurality of carry generators. 

30 11. An apparatus as defined in Claim 3, further comprising 

means for selecting a largest one among the MPC(x,y,d) values 
for R(x,y) and providing a d value yielding the largest MPC 
value as the disparity for R(x,y). 

35 12. An apparatus as defined in Claim 3, where said P-buffer 

includes means for storing Ix *Wy* (Sr+l) P values, wherein Ix 
is the number of pixels in a row in the reference and the 
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search image. 

13. An apparatus as defined in Claim 6, where said V-buffer 

includes means for storing Ix* (Sr+1) V values, wherein Ix is 

the number of pixels in a row in the reference and the search 
image . 



wo 99/53681 



0 ^/TfeR99/00'l 74^ 



1/14 



O 
< 
g 

o 



O 
O 



wo 99/53681 



PCT/KR99/00174 



2/14 



<< 



CM 

• r— f 



O 
pc5 



CO 



P3 



PCT/KR99/00174 



3/14 



CO 

•I — I 



CO 



wo 99/53681 



PCT/KR99/00174 



4/14 



HE 



+ 



> 
II 



•1 — I 



• I — I 



wo 99/53681 



09/6Z35tfe 

PCT/KR99/00174 



5/14 



START J 



Fig. 5 



,S20 



for d = 0 to Sr 
for x = 0 to Ix-l-d 

wy 

V(x,y,d) = X^T(x.y + i.d) 



End 3 ~ 



S85 
1 




.S30 



S40 



S60 



-S50 



y=y+i 



|x = X-H| I d+Q.Max_MPC=-l 



S80 




S95 



SlOO 



|d = d+l| 



if =wx then MPC(x.y,d)=_Z;v(x + i.y,d) 
else if x> = lx-dwx then ^~ 
MPC(x.y.d)=0 

V(x-l-wx,y+l,d)=V(x-l-wx.y,d)+T(x-l-wx,y + l + wy,d)-T(x-l-wx,y-wy,d) 

M?C(x,y.d) =MPC(x-l .y.d) +V(x + wx,y ,d)-V(x-l-wx,y.d) 
V(x-l-wx.y+l.d)=V(x-l-wx.y.d)+T(x-l-wx.y + l+wy.d)-T(x-l-wx,y-wy.d) 



S130 




DM(x.y) = d 
Max_MPC = MFC (x.y .d) 



"^S140 



wo 99/53681 



PCT/KR99/00174 



6/14 




• r-H 



wo 99/53681 



PCT/KR99/00174 



7/14 



I 



P-> (M 



Oh <N 



T 



• I— I 



wo 99/53681 



PCT/KR99/00174 



8/14 



PQ 



^1 



wo 99/53681 



09/6235lb 

PCT/KR99/00174 



9/14 




o Q 



>,00 



wo 99/53681 



PCT/KR99/00174 



10/14 



Fig. 8b 



D_P UNIT 



211 




T(4) T(2) T(0) 
T(5) T(3) T{1) 



wo 99/53681 



PCT/KR99/00174 



11/14 



^ 



O 






5 






W MP 
COUNT 
UPDAT 




CO 


CO >c 


> ; 


> > 



00 



X 

'P CD 


o 




X3 



CM ' 



CD 



•I — I 



c 


J 
















^ o 




o 








wo 99/53681 



09/6235U 

PCT/KR99/00174 



12/14 



Fig. 10 



Vo(2) 



Vo(l) 




wo 99/53681 



PCT/KR99/00174 



13/14 




wo 99/53681 



PCT/KR99/00174 



14/14 



Fig. 12 



C&A(n)_CELL (321) 



(231-2), 



\ 



Sb(n-l)- 
Sa(n-l)- 

Sb(0) - 
Sa(0) - 

Ib(5) 
Ia(5) 
Ib(4) 

Ia(4) 
Ib(3) 

Ia(3) 
Ib(2) 

Ia(2) . 
Ib(l) 

la(l) 
lb(0) 

la(0) . 



j H>>-4 



CG 



GG 



CG 



CG 



CG 



(231-1) 



CG 
i-Vdd 



-S(n) 
►S(n-l) 

-S(O) 
-V(5) 
-V(4) 
-V(3) 

- V(2) 

- V(l) 
-V(0) 



(231-3) 



ATTORNEY/DOCKET NO: 

Declaration for Patent Application and Appointment of Attorney 

As a below named inventor, I hereby declare that my residence, post office address and citizenship are as stated below next to my 
name; I believe that I am the original, first and sole inventor (if only one name is listed below) or an original, first and joint inventor (if plural 
names are listed below) of the subject matter which is claimed and for which a patent is sought on the invention (Design, if applicable) entitled: 

METHOD AND APPARATUS FOR MEASURING SIMILARITY USING MATCHING PIXEL COUNT 
the specification of which (check one): 

O is attached hereto, or was filed on: 

Number: pcT/KR99/0pl 74 

and (:f applicable) was amended on: 
I hereby state that I have reviewed and understand the contents of the above-identified specification, including the claims, as amended by any 
amendment(s) referred to above. I acknowledge the duty to disclose information which is material to patentability as defined in Title 37. Code 
of Federal Regulations, §1.56. I hereby claim foreign priority benefits under Title 35, United States Code §119 of any foreign application(s) for 
patent or inventor's certificate listed below and have also identified below any foreign application for patent or inventor's certificate having a 



as U.S. Application Number or PCT Intemational Appiic; 



1 Prior Foreign Application(s) 


PRioRrrY Claimed 


1 Number 


Country 


Day/Month/Year Filed 


Yes 


No 


1 -^cic 




13/04/1998 


X 















□ Additional Priority Application(s) Listed on Following Page(s) 



1 HEREBY CLAIM THE BENEFIT UNDER TITLE 35 U.S. CODE §1 19(E) OF ANY U.S. PROVISIONAL APPLICATIONS LISTED BELOW. 


Application Number 


Day/Month/Year Filed 











□ Additional Provisional Application(s) Listed on Following Page(s) 

1 hereby claim the benefit under Title 35, United States Code, §120 of any United States application(s) or PCT intemational 

- application(s) designating The United States of America listed below and, insofar as the subject matter of each of the claims of this application is 

- not disclosed in that/those prior application(s) in the manner provided by the first paragraph of Title 35. United States Code. §112,1 acknowledge 
the duty to disclose information which is material to patentability as defined in Title 37. Code of Federal Regulations. §1.56 which became 

" ava lable between the filing date of the prior application(s) and the national or PCT intemational filing date of this application: 



Application plumber 



Filing Date 



Status - Patented, Pending or 
Abandoned 



o Additional US/PCT Priority Application(s) listed on Following Page(s) 

I hereby declare that all statements made herein of my own knowledge are true and that all statements made on information and belief 
are believed to be true; and further that these statements were made with the knowledge that willful false statements and the like so made are 
punishable by fine or imprisonment, or both, under section 1001 of title 18 of the United States Code and that such willful false statements may 
jeopardize the validity of the application or any patent issued thereon. 

POWER OF ATTORNEY: I (We) hereby appoint as my (our) attorneys, with full powers of substitution and revocation, to prosecute 
I this application and transact all business in the Patent and Trademark Office connected therewith: J. Ernest Kenney, Reg. No. 19,179; Eugene 



/ Mw, Reg. No. 25.893; Richard E. Fichter, Reg. No^ 
ixfeenjamin E. Urcia, Reg. No. 33,805; and 



6.382 ; Thomas J. Moore, Reg. No. 28,974; Joseph DeBenedicris, Reg. N o. 28,502; 



I(we) authorize my(our) attorneys to accept and follow instructions from regarding any 

matter related to the preparation, examination, grant and maintenance of this application, any continuation, continuation-in-part or divisional 
based thereon, and any patent resulting therefi-om, until I(we) or my(our) assigns withdraw this authorization in writing. 



Send correspondenc 



BACON & THOMAS^L] 

"^25 Slaters Lane""^TffiTKbr 



Telephone Calls to: (703) 683-0500 



Full Name OF First OR SOLE INVENTOR ^^^j^ HYOUNG GON 


CITIZENSHIP KOREA 


Residence Address 

12-202, SEOCHOWOOSUNG APT., 1336, SEOCHO- 
DONG , SEOCHO-GU , SEOULjk' J*37,-070 , 
REPUBLIC OF KOREA AT ^ 1/ 


Post Office Address is the same as Residence Address unless 
otherwise shown below 


^"^^ SEPTEMBER 30, 2000 


Signature j / i , 
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O See following page(s) for additional joint 



Attorney/Docket No: 

Continuation of Declaration for Patent Application and Appointment of Attorney 



Page 



Prior Foreign Application(s) (35 (JSC §119 


) 


Priority Claimed 


Number 


Country 


Day/Month/Year Filed 


Yes 


No 























Prior Provisional Applicat 


ONS 35 U.S. Code §1 19(E) 


Application Number 


Day/Month/Year Filed 











Prior U.S. or PCT International Applications (3 


5 U.S. Code §120) 


Application Number 


Filing Date 


Status - Patented, Pending or Abandoned 















'"^"°|t"'''''°' SUNG MIN 


Citizenship 

KOREA 


Residence Address 
SEOUL BUILDING, 114-31, UNI-DONG, CHONGRO- 
KUj^^SEOOT^ 110-350, REPUBLIC OF KORE_^^- 


Post Office Address is the same as Residence Address unless otherwise 
shown below 


SEPTEMBER 30, 2000 


SIGNATURE ^^^.C^ . 




Full Name of Joint Inventok^ ) 

^^HKJ AHN, SANG CHUL 


CITIZENSHIP ^Q^g^ 


Residence Address 

204-805, RUWON APT., DANSAN-DONG, YOUNGDEUNC 
-KU, SEOUL, 150-045, REPUBLIC OF KOREA 


Post Office Address is the same as Residence Address unless otherwise 

,-j^OWN BELOW 

/ 


Date 

SEPTEMBER 30, 2000 


0,1 An 




FuLLNA.EOF^._T,Mve,T^j KIM, NAM KYU 


Citizenship 

KOREA 


Residence Address 
107-302, SIYOUNG APT., CHUNGKYEljDONG, 
NOWON-KU, SEOUL, 139-785, U 
RRPTmT.Tr np Tcnp-Ra / rj/i 


Post Office Address is the same as Residence Address unless otherwise 
shown below 


SEPTEMBER 30, 2000 


Signature (. '/^ |^ ^ 




Full Name of Joint Inventor 


Citizenship 


Residence Address 


POST Office Address is the same as Residence Address unless otherwise 

shown BELOW 




Signature 



□ See following pages for additional joint inventors/priority applications. 
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